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EXECUTIVE SUMMARY 



There is growing concern that the nation’s most effective teachers are not working in the 
schools with the most disadvantaged students (Goldhaber 2008; Peske and Haycock 2006; 
Tennessee Department of Education 2007; Sass et al. 2010; Glazerman and Max 2011). 
Policymakers at the federal, state, and local levels have considered a range of policies for helping 
struggling schools attract and retain effective teachers. One goal of such policies is to improve 
the access that disadvantaged students have to top teachers. Strategies for advancing this goal 
include alternative teacher preparation and certification, recruitment bonuses for serving in hard- 
to-staff schools or subjects, intensive mentoring and professional development, and performance- 
based pay. These strategies have been implemented with funding from the federal government, 
state and local governments, and nongovernment sources. Some have also been implemented in 
the context of research intended to gauge their effectiveness (Glazerman et al. 2006; Glazerman 
et al. 2010; Glazerman and Seifullah 2010; Springer et al. 2011). But to date, there is little 
rigorous evidence of any of these strategies demonstrating clear success in raising student 
achievement in the U.S. 

This report describes the implementation and intermediate impacts of an intervention 
designed to provide incentives for a school district’s highest-performing teachers to work in its 
lowest- achieving schools. The report is part of a larger study in which random assignment was 
used to form two equivalent groups of classrooms organized into teacher “teams” that are 
composed of teachers in the same grade level and subject (math, reading, or both in the case of 
an elementary school grade). Teams were assigned to either a treatment group that had the 
chance to participate in the intervention described below and or a control group that did not. 
Intermediate outcomes, measured for both the treatment and control teams, include the mix of 
teachers who make up the team, the climate of collaboration and cooperation in the team, and the 
way in which resources are allocated within the teacher team. A future report will focus on the 
impacts of the intervention on student achievement and other outcomes like retention. 

A Test of Using Transfer Incentives for Highest-Performing Teachers 

One strategy that has not been studied in sufficient detail is the use of monetary recruitment 
incentives targeted specifically to teachers who have demonstrated success in raising student test 
scores (“value added”). The U.S. Department of Education’s Institute of Education Sciences 
(lES) has sponsored the current study that tests the effectiveness of an intervention based on this 
strategy. The intervention is known to participating school districts as the Talent Transfer 
Initiative (TTI). The TTI offers $20,000 to highest-performers within certain categories of 
teachers if they agree to transfer and remain for at least two years in one of the selected low- 
achieving schools in the district. 
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The intervention was designed to proceed as follows. The first step is to conduct a value- 
added analysis of student test scores to identify the highest-performing teachers, defined as the 
top 20 percent based on a value added measure of teachers in tested grades and subjects in each 
district.^ The TTI relies on at least two years — and typically three, depending on district data — of 
student achievement growth data for each teacher. 

The second step is to classify schools as “potential receiving” or “potential sending” 
schools. Potential receiving schools are those with the lowest achievement in the district and 
which the district leaders intend to help through the intervention. As discussed below, selected 
teaching positions in these schools, or “vacancies,” are eligible for the transfer incentive. The 
rest of the schools in the district, with rare exceptions for special schools that are exempted, are 
potential sending schools. 

The third step is final selection and recruitment of eligible sending school teachers and 
receiving school principals. An implementation team determines which of the highest- 
performing teachers (identified in the first step) are in potential sending schools and offers them 
a series of transfer incentive payments, totaling $20,000 over two years, to transfer into and 
remain in one of the receiving schools in their district. The offer is made to these teachers, 
known as “transfer candidates,” in the spring, at which point they are invited to apply to the 
program. At the same time, principals of potential receiving schools are invited to an information 
session and asked to identify likely teaching vacancies in the targeted grades and subjects. 
Selected teaching vacancies are then designated to be eligible for the transfer incentive. A site 
manager in each district matches transfer candidates to principals with eligible receiving school 
vacancies, assisting both teachers and principals in arranging interviews to fill the targeted 
vacancies. 

Next, applicants must interview with and be accepted by the principal at the receiving school 
and then voluntarily transfer in order to qualify for the transfer incentive. In order to improve the 
probability of finding a successful match, the study implementation team worked with each 
district to finalize offers and acceptances by early summer. 

Finally, the transfer teachers are given a half-day orientation just before the start of the 
school year. Because they are selected on the basis of their performance in the classroom, it is 
assumed that they do not require additional formal support. To facilitate the transition, however, 
the site manager provides informal support and answers any questions as needed during the two 
school years that constitute the intervention period. During that time, teachers who remain in 
their originally assigned positions receive incentive payments at the end of each semester, in 
December and June. 



^ Value-added analysis is the statistical approach intended to determine the unique contribution each teacher 
makes to student achievement, holding constant factors that are outside the teacher’s control. The cutoff for a 
teacher to be deemed “highest performing’’ in a district was typically 20 percent, but the cutoff varied across districts 
and pools (middle school math teachers, middle school English/language arts teachers, and elementary multiple- 
subject teachers). It was lowered to 18 percent for elementary teachers in two districts and raised to 25 percent for 
middle school teachers in another district. Another district set the cutoff for middle school at 23 percent for math 
teachers. 
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Teachers in the highest-performing group who are already teaching in low-achieving schools 
are not eligible to transfer under the program. Instead, they automatically qualify for a retention 
bonus of $10,000, which is also paid in installments over two years, as long as they remain in 
their schools. 

The intervention was implemented in elementary and middle schools in a pilot district 
starting in 2008 and expanded to include the pilot site and six other districts in five states in 2009 
(cohort 1). Three more districts were added in 2010 (cohort 2). This report focuses on the seven 
cohort 1 districts; a future report will incorporate information from all 10 districts in cohorts 1 
and 2. 

Research Questions and Study Design 

The study addresses implementation and impact. This report focuses primarily on the 
implementation and intermediate impacts, the first two questions listed below. The third question 
listed below will be the focus of a future report. 

• What was the TTI implementation experience with respect to the teacher 
recruitment process? 

• What were the teacher placement results and intermediate impacts of TTI? For 
example, who filled the vacancies compared to those who would have filled the 
vacancies in the absence of the intervention? How did the intervention affect 
collaboration? How did it affect the allocation of resources within the school, 
such as assignment of students to teachers, teacher mentoring, and teacher 
leadership? 

• What was the impact of TTI on teacher retention and student achievement? 

The methods for answering these questions include descriptive tabulations (for 
implementation questions) and causal analysis (for impact questions). The causal analysis relies 
on a random assignment procedure discussed next. 

Random Assignment 

To answer the impact questions, we implemented a randomized controlled trial. The study 
focuses on teacher “teams” composed, as mentioned above, of all the regular classroom teachers 
in a given grade level and subject within a school, starting with teams that had at least one 
teaching vacancy. For elementary school grades, in which teachers are often responsible for both 
math and reading instruction, we considered the whole grade level to be a team. For middle 
school grades, teacher teams were composed of either math or English language arts (ELA) 
teachers. Eor example, all teachers responsible for teaching seventh-grade math in the same 
school would make up one team. All teachers in the school who were responsible for eighth- 
grade language arts would be considered another team. We randomly assigned teacher teams to 
either a treatment status, defined as the chance to fill the team’s vacancy with a TTI teacher, or a 



^ This question is not addressed in the current report but will be addressed in a future report. 
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control status, in which vacancies were filled through whatever process the school would 
normally use. 

This process created two groups that were, on average, similar in terms of student 
characteristics and school context. The only systematic difference between the two groups was 
whether the person filling the vacancy was eligible for the $20,000 transfer incentive. Comparing 
outcomes for these groups will generate unbiased estimates of the impact of TTI on student 
achievement and other outcomes. 

We expect much of the impact of TTI to operate through the teachers who filled the 
vacancies in the treatment and control teacher teams. We refer to them as “focal” teachers. 
Therefore, in addition to the team-level analysis, we are interested in the comparison between 
focal treatment and focal control teachers. We refer to the other teachers on the teams as “non- 
focal” teachers. 

Data Collection 

The data for the study come from survey and administrative records data as well as program 
implementation records. Surveys were conducted with teachers who were transfer candidates, 
regardless of whether they transferred (“Candidate Survey”); with teachers in teams with 
vacancies, including both treatment and control teams of teachers (“Teacher Background 
Survey”); and with their principals (“Principal Survey”). The administrative data include student 
test scores linked to teachers, demographic data, and teacher rosters. All surveys described in this 
report were administered during the 2009-2010 school year. 

We obtained response rates of 83, 80, and 95 percent on the Candidate Survey, Teacher 
Background Survey, and Principal Survey, respectively. We received teacher roster data for 100 
percent of the schools in the study at baseline (fall 2009) and followup (fall 2010). 

Study Sample 

We selected school districts that were large and economically diverse. They had to have at 
least 40 elementary schools, at least 10 of which had to be low-poverty schools and at least 15 of 
which had to be high-poverty schools. Low- and high-poverty schools were defined as having 
less than 40 percent or more than 70 percent of students eligible for free or reduced-price lunch 
(FRL), respectively. In addition to these quantitative criteria, we selected districts on the basis of 
a variety of qualitative factors related to feasibility of implementation, including test score 
availability, data quality, hiring/transfer practices, and the local political environment. The 
resulting set of districts was not a random sample of a well-defined population of districts, so 
findings from this study cannot necessarily be generalized to other districts. 

While we excluded school districts in which existing or planned teacher incentive programs 
would have duplicated the intervention under study, we did come across some existing policy 
initiatives in each of the seven participating school districts. These programs included 
performance incentives and signing bonuses for teachers. In each case, we determined that the 
existing programs were different enough, isolated to a few schools that could be excluded from 
our study, or involved small enough dollar amounts that they would not interfere with the study 
design. Teachers and schools receiving more than $5,000 — an arbitrary threshold we used to 
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identify substantial bonus programs — were excluded in order to avoid complicating the study by 
changing the effective differential in the TTI transfer incentives relative to the counterf actual. 

The sample for this report comprises seven districts in five states. Five of the seven districts 
are county districts, so they include urban centers as well as suburban and rural areas. Together, 
the seven districts range in size from 55 to 218 elementary and middle schools. Hispanic students 
make up the majority of students in two of the districts. African American students make up the 
majority of students in one district. Another district has a majority white student body, and the 
remaining three do not have a majority of any one racial/ethnic group (Hispanic, African 
American, white, or other). 

Working with each district, the implementation team divided the elementary and middle 
schools in the district into potential sending or potential receiving schools according to academic 
ranking; the potential receiving schools were those targeted to benefit from the intervention. 
Schools were ranked by their students’ average prior achievement level, which was determined 
by the prior three years of achievement data or by the past year’s achievement data, depending 
on the district leaders’ preferences."^ The lowest-ranking schools were designated as potential 
receiving schools, and the rest were potential sending schools. Some schools were removed from 
both pools and referred to as exempt schools because they served a special population of students 
or were already implementing a program that was meant to address the problem that TTI is 
intended to address. In the end, 21 percent of the schools were classified as potential receiving 
schools, 70 percent were potential sending schools, and the remaining 9 percent were exempt. 

The potential receiving schools were more disadvantaged than the potential sending schools, 
as measured by the proportion of students eligible for FRL. In the elementary schools, 78 percent 
of students in the average receiving schools were eligible for FRL, compared with 64 percent of 
students in the sending schools, a statistically significant difference of 14 percentage points.^ In 
middle schools, the difference is also statistically significant, equal to 20 percentage points (74 
versus 54 percent). 

Not every potential sending school had a teacher transfer out, and not every potential 
receiving school had a teacher transfer in. To become an actual receiving school, principals of 
potential receiving schools first had to voluntarily submit at least one vacancy to the study team 
for randomization and it had to have been assigned by the researchers to the treatment group. 



Achievement data from the year prior to the implementation of TTI were used for all but two districts, where 
three prior three years of achievement data were used. 

^ The discussion of “significant differences” here and throughout the report refers to statistical significance. 
We used a 0.05 significance level, which means that a significant difference is highly unlikely (less than five percent 
of the time) to be observed in a sample if the population difference was zero. Statistical significance does not imply 
that the difference is meaningful to policy, nor does a lack of statistical significance imply that the difference is not 
meaningful for policy. 
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Finally, there had to be at least one eligible TTI candidate who applied to, interviewed at, and 
successfully transferred to one of the receiving school positions.^ 

Across the seven districts, teachers who transferred through TTI came from 51 out of 512 
(10 percent) of the potential sending schools. This percentage is relevant for districts concerned 
that a transfer program like TTI might be disruptive to many of its sending schools. The finding 
that 90 percent of the schools identified as potential sending schools did not lose a single teacher 
suggests that sending schools in these districts did not face widespread disruption as a result of 
the intervention. 

The TTI teachers transferred into 48 potential receiving schools. These schools represent 91 
percent of the 53 schools with at least one treatment team. The other 101 potential receiving 
schools included 68 non-study schools (those with no vacancies submitted for random 
assignment) and 33 schools with only control teams. 

The impact study focuses on the teacher teams in the 53 schools with at least one team 
assigned to the treatment group and the 33 schools with teams assigned to the control group. 
Each team included all teachers in the grade and subject of the randomly assigned vacancy. 
Because schools could submit more than one vacancy for randomization, some schools included 
multiple study teams. The final sample of treatment and control teachers and schools for this 
report consisted of 451 teachers in 124 teacher teams. Eighty percent of the teachers were in 
elementary schools (grades 3 to 5), and the other 20 percent were in middle schools (grades 6 to 
8 ).’ 



The differences between the treatment and control teams in terms of baseline characteristics 
such as student background and teacher professional background were not statistically 
significant. We examined students’ prior achievement, race/ethnicity, English language learner 
status, ERE status, and special education status. Eor teachers who were already teaching in the 
study teams before TTI (i.e., not including TTI transfers and other new hires), we examined their 
experience level in the profession, in the district, and in their schools as well as their degrees and 
certifications. With regard to teachers’ personal characteristics, we found no statistically 
significant differences at baseline except for marital status (45 percent of treatment teachers 
versus 65 percent of control teachers were married) and presence of children in the home (40 
percent of treatment teachers versus 56 percent of controls). 

Summary of Findings 

The main findings from this first phase of the study primarily focuses on eligible transfer 
teachers, teachers who choose to transfer, and the experiences of principals and teachers in the 
study schools. In addition, this report includes intermediate outcomes related to some measures 
of collaboration and school resource allocation. 



® A team and a school were in the study if a vacancy was submitted and randomly assigned even if no teachers 
transferred to that school. Random assignment occurred after principals voluntarily submitted vacancies (consented). 

^ There were 137 vacancies in 124 teacher teams because 11 teams had 2 vacancies, and one team had 3 
vacancies. In the treatment group alone, there were 6 teams with 2 vacancies and no teams with more than 2. 
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• Filling vacancies by using transfer incentives was shown to be feasible. The 

implementation of TTI demonstrated that it is possible to implement a transfer 
incentive program as designed for this study. The highest-performing teachers were 
identified by using value added analysis in approximately the first three months of the 
calendar year, followed by two to three months of intensive recruitment of receiving 
schools and transfer candidates, after which 90 percent of vacancies were filled. 

• A large pool of candidates was needed to yield the desired number of successful 
transfers. We found that an average of six percent of each district’s highest- 
performing teachers in non-low-performing schools ultimately transferred to low- 
performing schools. Over one-third (38 percent) of eligible transfer candidates 
attended an information session, and 24 percent completed an application. There were 
1,012 candidates identified for the 63 who ultimately transferred, a ratio of 16 
candidates per slot. 

• Transfer teachers came from sending schools and classrooms with significantly 
different characteristics, on average, than the receiving schools they transferred 

to. The average transfer was from a school in the 60th percentile for average test 
scores to one in the 18th percentile. In districts where data were available, we found 
that 64 percent of the transfer teachers’ students were low-income, defined as eligible 
for FRL, versus 89 percent of their students after they transferred. Before these 
teachers transferred, their average student scored in the 48th percentile on prior math 
tests compared to the rest of the district, but after they transferred, their average 
student had scored in the 32nd percentile on prior math tests compared to the rest of 
the district. 

• The transfer teachers were more experienced than teachers normally tapped for 
the positions filled by TTI teachers. The average difference in teaching experience 
was five years. There was also a significant difference in the percentage of teachers 
with a post-graduate (master’s or doctoral) degree (48 percent of focal treatment 
teachers and 21 percent of focal control teachers). 

• Survey data suggest that there are no statistically significant differences in 
school climate or in how students were assigned to teachers as a result of TTI. 

Treatment-control differences in how principals rated their teaching teams in terms of 
collaboration and the sharing of ideas were not statistically significant. Treatment- 
control differences in the patterns of student characteristics and in principal or teacher 
survey responses regarding the assignment of students to classrooms were not 
statistically significant either. 

• TTI transfers used less mentoring but provided more mentoring than their 
control group counterparts. Treatment focal teachers were less likely to have a 
mentor than were control focal teachers (39 versus 66 percent), but they spent more 
time mentoring their colleagues (25 more minutes per week, on average). 

Detailed Findings on the Teacher Recruitment Process 

How Were Transfer Candidates Identified and Recruited? 

Transfer candidates were identified through value-added analysis of teachers in all schools 
in the district. The analysis was conducted separately for the three grade-subject pools within 
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each district: middle school math teachers, middle school ELA teachers, and elementary 
multiple-subject teachers (grades 3 to 5). Teachers in the top 20 percent of value-added ranking 
in their grade-subject pool were considered to be the highest performing teachers in the district 
and were designated as transfer candidates. The 20 percent cutoff was chosen so as to be 
selective while providing enough transfer candidates to yield enough program applicants to fill 
all of the vacancies identified in the receiving schools. 

The selective nature of the 20 percent cutoff is reflected in the estimates of the performance 
difference between transfer candidates and all other teachers. We found that transfer candidates, 
who were identified as highest performing, contributed 16 percent of a standard deviation more 
to student reading achievement in a year than did the average teacher in the district; they also 
contributed 24 percent of a standard deviation more to math achievement. Value-added 
differences of this magnitude are consistent with an average transfer candidate raising a median 
student 6 percentile points in language arts and 1 1 percentile points in math relative to all other 
teachers in the value-added analysis. 

To assess the adequacy of the 20 percent cutoff in identifying a large enough group of 
transfer candidates, we conducted a pilot study of TTI in one of the districts starting in 2008. 
From that experience, we estimated that we would need at least 10 transfer candidates per 
position that the transfer incentive was meant to fill. For the seven districts that implemented the 
TTI in 2009, 1,385 transfer candidates were identified (including 373 who turned out to be 
ineligible), and 63 positions were filled, at a ratio of almost 22 to 1. 

Transfer candidates were recruited by the site manager in each district. The site managers 
held information sessions separately for potential receiving-school principals and for transfer 
candidates. They also followed up with every teacher and principal by telephone, email, or 
regular mail. Once vacancies were identified and the teaching teams were assigned to a treatment 
or a control group, the site manager encouraged transfer candidates to interview for specific TTI 
incentive-eligible openings and facilitated the scheduling of those interviews. 

According to site managers, the success of the intervention depended a great deal on 
recruiting transfer candidates early in the process, so they worked with principals to determine, 
as early as possible, when they would have a teaching vacancy eligible for the transfer incentive. 
The bulk of vacancies in the seven TTI districts were identified and filled in May and June 2009; 

o 

75 percent of vacancies were identified, and 70 percent were filled in these two months. The 
time between identifying and filling a vacancy varied, with site managers reporting that 
vacancies were filled as quickly as two days after being assigned. Among the 70 vacancies 
assigned to treatment, 63 (90 percent) were filled with a TTI candidate by the end of the summer 
recruitment season. 

Transfer candidate recruitment and placement, including interactions with receiving- school 
principals, took place between March and September 2009, and it was the most labor-intensive 



In this case, “identifying” a vacancy meant assigning it by lottery to the treatment group. Since the study 
required pairs or groups of vacancies in the same grade to be considered at the same time, identifying TTI positions 
may have taken longer than it would have in the absence of a study. In addition, the need to identify vacancy pairs 
matched on grade and subject within the same time frame limited the number of vacancies relevant to the study. 
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part of implementing the intervention. The whole process required the efforts of a site manager 
who spent about one-third to one-half of a full-time equivalent per district, depending on their 
experience, the number of transfers, and the responsiveness of the teachers and principals. 

How Did Teachers and Principals React to the Talent Transfer Incentives? 

We measured the behavioral response of teachers by observing the percentages of the 1,012 
transfer candidates we identified who came to an information session, applied to the program, 
interviewed for a position, and accepted a transfer offer from receiving school principals. We 
refer to these percentages as “take-up rates.” 

Figure ES.l shows the take-up rates by grade span and subject. Transfer candidates were 
invited to a “recognition event” that also served as an information session and a recruitment 
opportunity; an average of 38 percent of candidates attended the event. Approximately one- 
quarter of all transfer candidates applied to the program, which involved completing a one-page 
online form; 13 percent interviewed for at least one vacancy, and 6 percent ultimately 
transferred. Given that 90 percent of the designated slots were filled, these transfer rates could be 
a useful guide for implementing an intervention like TTI in the future. 

Which Teachers Apply to Transfer and Do So Successfully? 

We examined both the pre-transfer student characteristics and the value-added scores of 
transfer candidates who did and did not apply to TTI to determine whether there were any 
patterns. We also conducted a multivariate (regression) analysis of transfer applications to better 
estimate the relationship between the transfer candidates’ circumstances and their probability of 
applying to transfer. The analysis showed that transfer candidates with a higher percentage of 
disadvantaged students (measured by FRL status) and transfer candidates with the very highest 
value-added rank (in the top 10 percent, as opposed to all in the top 20 percent) were 
significantly more likely to apply to TTI than were the rest of the transfer candidates. If we look 
at actual transfers as the outcome, only transfer candidates who were satisfied with their current 
school students were significantly more likely to transfer. No other characteristics of candidates 
or their students were significantly associated with whether the candidate transferred. 
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Figure ES.1. Take-Up Rates Among Transfer Candidates in the Seven Cohort 1 Districts 




Source: TTI program records. 

Note: Transfer candidates are the teachers we identified as highest performing in their pooi and 

within their district who were aiso teaching in potentiai sending schoois. We considered three 
poois: eiementary, middie schooi Engiish/ianguage arts (ELA), and middie schooi math. 

Where Do TTI Transfers Come From? 

If we establish a discrete and somewhat arbitrary dividing line between low-achieving 
schools, which the intervention is meant to help, and all other schools, it is possible for a teacher 
to transfer from a school just above the threshold to one just below it. We call these moves 
“lower-contrast transfers” because the difference in rank between the sending and receiving 
schools is small. Lower-contrast transfers might be counterproductive if the sending school is 
itself in need of strong teachers and has difficulty filling the vacancy created by the transfer. 

One way to address the question, “where do transfers come from?” is to compare the 
ranking of the sending schools that the transfer teachers left to the receiving schools to which 
they transferred. In terms of achievement rank, the average transfer represents a 42-percentile 
point change from the 60th percentile school (with 100 percent being highest performing in the 
district) to the 18th percentile school. In terms of income, the average transfer represents a 
change from the 55th to the 18th percentile of percent FRL. 

Another way to address the question is to compare the characteristics of students taught by 
TTI transfer teachers in their sending schools before they transfer to the students in their 
receiving schools after they transfer. On average, we found statistically significant differences 
between students taught by TTI transfer teachers in their original schools and students they 
taught in the schools to which they transferred. Table ES.l shows the average student 



ES-10 




Executive Summary 



Moving Teachers 



characteristics before and after transfer for the 33 transfer teachers who came from the four 
districts for which we had detailed individual student data for both sending and receiving 
schools. (We cannot say whether we would have achieved the same result for the other three 
districts had data been available.) 

On average, TTI teachers moved to classrooms with a lower percentage of white students 
(12 versus 30 percent) and a higher percentage of Hispanic students (42 versus 31). They also 
moved to classrooms with a higher percentage of low-income students, as measured by FRL 
eligibility (89 versus 64 percent). The percentage of students who were designated as special 
education students decreased by 8 percentage points from 19 to 11, although we could not 
distinguish from the district data how much of that decrease could have been due to a drop in the 
percentage of gifted and talented students, who were labeled by two districts as special 
education. 

Table ES.1. Characteristics of Selected Transfer Teachers’ Students Before and After Transferring 



Characteristic of Average Student 
(percentages unless noted) 


In Sending 
Schools 
2007-2009 


In Receiving 
Schools 
2009-2010 


Difference 
(Receiving 
Minus Sending) 


P-value 


Race/Ethnicity 

White 


30.1 


12.1 


-18.0* 


0.002 


African American 


32.2 


40.1 


7.9 


0.091 


Hispanic 


30.8 


41.8 


11.0* 


0.001 


Low income (percent FRL) 


63.6 


89.3 


25.7* 


0.000 


Special education® 


19.0 


11.2 


-7.9* 


0.032 


Limited English proficient 


12.9 


7.8 


-5.1 


0.125 


Average reading score*’ 


-0.11 


-0.39 


-0.28* 


0.021 


Average math score*’ 


-0.06 


-0.47 


-0.41* 


0.000 



Source: Administrative data. 

Note: Data pertain to a subgroup consisting of four districts that provided student-ievei data. N = 33 teachers 

who transferred in the four districts and for whom detaiied student data were avaiiabie. Because of 
missing data, the sampie size was 26 teachers for FRL and LEP, 25 for reading scores, and 23 for math 
scores. Not aii teachers taught both math and reading. 

® The speciai education category in two of the four districts inciudes gifted students. 

'^Average reading and math scores are given in fraction of a standard deviation computed within district and grade. 

* Difference is statisticaliy significant at the 0.05 ievei, using a two-sided test based on the teacher sampie. 

Test score differences between the transfer teachers’ sending and receiving schools were 
also statistically significant for this group of teachers. The average student in the transfer 
teachers’ classrooms scored 0.11 standard deviations below the district average in reading, 
placing them in the 46th percentile. The same teachers’ students in the schools to which they 
transferred scored 0.39 standard deviations below the district average, placing them in the 35th 
percentile. For math, the differences were -0.06 standard deviations (48th percentile) and -0.47 
standard deviations (32nd percentile). 
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Detailed Findings on Teacher Transfer Placement and Intermediate Outcomes 

Who Filled the Vacancies? 

The treatment group tells us who eventually filled the positions through the transfer 
incentive, but the control group tells us what might have happened in the absence of the 
incentive. Most treatment group vacancies (90 percent of the 70 positions assigned) were indeed 
filled through TTI, but 6 percent were filled outside the program. The remaining 4 percent of 
positions were “lost.”^ The 67 control group vacancies were filled through a variety of means, 
including new hires (21 percent), transfers (19 percent), and reassignments from within the 
school (27 percent). The remaining 31 percent of positions were either lost or filled by the same 
teacher who left (10 percent), or the status could not be determined (21 percent). 

As a result of this mix of teachers filling study positions, the treatment group teachers were 
more experienced, on average, than the control group teachers, but the differences were less than 
they might have been had 100 percent of the vacancies in the control teams had been filled by 
beginning teachers. The five-year difference in average experience in teaching (13 years versus 8 
years) was statistically significant. Treatment focal teachers were also more likely than control 
focal teachers to have an advanced degree (48 percent versus 21 percent) and were five years 
older on average. The comparison of teacher groups is shown in Table ES.2. 

What Was the Impact on Collaboration and Resource Allocation? 

One possible concern about an intervention that offers large stipends to teachers for having 
produced high value-added scores is that it could undermine collegiality and weaken 
collaboration and trust within the teaching team. Such phenomena are difficult to measure, but 
we posed a variety of questions to principals about these aspects of school climate. The Principal 
Survey asked respondents to rate each of their teaching teams from 1 to 5 on the level of 
collaboration, the extent to which teachers trust and mutually respect one another, and the extent 
to which teachers seek ideas from one another. They were then asked how this rating compared 
to the same dimension at the beginning of the school year and in the previous school year. 

We did not find evidence of a breakdown of morale or any significant impacts on the way 
that teachers worked together. There were no statistically significant impacts on principals’ 
opinions of the degree of collaboration, trust, and sharing of ideas within grade teams. 
Treatment-control differences in the principals’ ratings of teacher teams were not statistically 
significant for the three dimensions of school climate: the level of collaboration, the extent to 
which teachers trust and mutually respect one another, and the extent to which teachers seek 
ideas from one another. 



^ Positions were lost when class size increased, enrollment declined, teachers who had been laid off were 
recalled to their original positions, or when planned exits by retirees or transfers were canceled. 
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Table ES.2. Characteristics of Teachers Who Filled Treatment and Control Vacancies 
(Percentages) 



Characteristic 


Treatment Focai 


Controi Focai 


Treatment- 

Controi 

Difference 


Professional Background 


Years of experience in teaching (average years) 


12.9 


8.0 


4.9* 


Has a master’s or doctorate degree 


47.6 


21.1 


26.6* 


Has Nationai Board Certification 


23.0 


11.8 


11.2 


Transferred via TTI 


95.0 


0.0 


95.0* 


Personal Background 

Race/ethnicity 

White, non-Hispanic 


45.9 


55.3 


-9.4 


African American, non-Hispanic 


31.1 


25.0 


6.1 


Hispanic or Latino 


16.4 


17.1 


-0.7 


Age (years) 


42.3 


37.1 


5.2* 


Married or iiving with a partner 


60.3 


61.8 


-1.5 


Home owner 


82.5 


51.3 


31.2* 


Sample Size (number of teachers)® 


63 


41 





Source: 2009-201 0 Mathematica Teacher Background Survey. 



Note: Consistent with the study design, we compare aii focai teachers in the treatment group to aii 

focai teachers in the controi group, regardiess of how the positions were fiiied. 

® The treatment group has 63 teachers instead of the 70 assigned because three transfer teachers 
changed grades, one vacancy was iost, and three teachers were nonrespondents. The smaiier sampie of 
controi focai teachers resuited from a combination of survey nonresponse, indeterminate focai teacher 
status, or a combination of both. 

* Difference between treatment focai mean and controi focai mean is statisticaiiy significant at the 0.05 
ievei using a two-sided test. 

How Were Students Assigned? 

We hypothesized that a transfer incentive could change the way schools assign resources 
within the teacher team. For example, a principal might try to leverage the experience and .skill 
of a transfer teacher by assigning that teacher more difficult students and assigning the less- 
challenging students to the remaining teachers in the grade. Therefore, we sought to understand 
the relationship between students assigned to focal teachers and non-focal teachers and to 
compare the focal/non-focal difference between treatment teams and control teams. 

We used three data sources to examine student assignment differentials: (1) administrative 
data on student characteristics, (2) teacher perceptions in the Teacher Background Survey, and 
(3) principal perceptions from the Principal Survey. The administrative data provide objective 
information on a few easily observed traits. The teacher survey data are subjective, but they 
capture differences not only in demographic characteristics but in students’ behavioral 
challenges. The principal survey data are also subjective, but they allow us to focus specifically 
on the assignment process and allow respondents to tell us directly how they intended to assign 
students. 
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None of the treatment/control differences was statistically significant. When we look at the 
distribution of student characteristics in the focal teachers’ classrooms relative to their non-focal 
peers, we found no significant difference between the treatment group and the control group. We 
examined their students’ math and reading pretest scores, race/ethnicity, FRL status, English 
language learner status, and special education status. We also compared treatment and control 
teachers’ responses to direct survey questions about whether their own students were more or 
less challenging than those of their peers and found no statistically significant differences there 
either. Finally, the results from the Principal Survey did not reveal statistically significant 
differences in the method principals said they used to assign students to classrooms. 

How Were Mentoring Resources Allocated? 

A TTI teacher can create opportunities to assign students strategically, but another approach 
might be to realign mentoring resources. Presumably, a teacher with a high value-added ranking 
and years of experience would require less mentoring than the typical teacher who fills a new 
position. If mentors spent less time with teachers in treatment teams, then they might have more 
time to spend elsewhere. To measure this outcome, we asked teachers to report on whether they 
had a mentor, from whom they received mentoring, and how much time they spent with a 
mentor. 

Focal teachers in treatment teams received less mentoring than did focal teachers in control 
teams. As shown in Figure ES.2, treatment focal teachers (made up of TTI transfers and whoever 
else filled the targeted vacancies) reported having a mentor at a significantly lower rate than did 
control focal teachers (39 versus 66 percent, p-value = 0.007). The time spent with a mentor per 
week on average was 33 minutes and 58 minutes for treatment and control focal teachers, 
respectively (the difference was not statistically significant, p-value = 0.105). The peer teachers 
(non-focal), both treatment and control, reported receiving levels of support that were in between 
that of their teams’ focal teacher: 43 percent had a mentor with whom they spent an average of 
50 minutes per week. 

We did not find evidence that the impact on focal teachers was offset by an equivalent 
opposite impact on non-focal teachers, which would happen if the resources were simply shifted 
within the team. Figure ES.2 also indicates that TTI teachers reported receiving less mentoring 
from other teachers in the school than did their counterparts (23 percent vs. 45 percent, p-value = 
0.024), which suggests that the reduced use of mentors by TTI teachers largely reflects a 
reduction in the reliance on other teachers in the school. 

Were TTI Teachers Used in Mentoring or Leadership Roles or Given Other Duties? 

Yet another way for a school to take advantage of the expertise of TTI teachers might be to 
assign them additional duties or responsibilities. The design of the intervention did not require 
principals to create or require that teachers perform any special duties or roles as a condition of 
being hired or receiving the TTI bonus, but principals were not restricted from imposing such a 
condition or from simply assigning the teacher to, or requesting that the teacher fill, such a role. 
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Figure ES.2. Mentoring Support, by Treatment and Focai Status 




Has a Mentor 
(%) 



Time with Mentor 
(Minutes/Week) 



I Focal Treatment 
I Focal Control 

Non-Focal Treatment 
I Non-Focal Control 



15 14 



17 







Another Teacher 
(%) 



Coach or Facilitator 
(%) 



Source of Mentoring 



Source: 2009-201 0 Mathematica Teacher Background Survey. 

Note: N = 62 treatment focal, 41 control focal, 115 treatment non-focal, and 113 control non-focal 

teachers. 

* Treatment-control difference (within focal status) is statistically significant at the 0.05 level using a two- 
sided test. 

We found that treatment focal teachers provided more mentoring to their peers than did 
control focal teachers (25 minutes versus less than one minute per week spent providing such 
assistance). The differences were statistically significant. Of the non-focal teachers, 18 and 20 
percent of treatment and control group teachers reported providing mentoring for just over half 
an hour per week. These differences for non-focal teachers were not statistically significant, so 
there is not enough evidence that the amount of mentoring provided by the treatment focal 
teachers resulted in an offsetting decrease in the amount of mentoring provided by their peers. 
None of the treatment/control differences in the rates at which focal or non-focal teachers played 
leadership roles were statistically significant. 

Next Steps 

A future report will present estimates of the impacts of TTI on achievement test scores and 
on the retention of the highest-performing teachers. That report will also update the findings 
from this report with findings from a second cohort of school districts, which is composed of a 
greater percentage of middle schools and will increase the overall sample size. 
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